System for certificating and synchronizing virtual world and physical world

ABSTRACT

A device or space existed in a physical space is registered in the form of a digital object in a virtual space, a digital twin service is provided through connection between an offline device or space and the digital object in the virtual space.

This application claims the priority benefit of Korean PatentApplication No. 10-2020-0075913, filed on Jun. 22, 2020, Korean PatentApplication No. 10-2020-0075914 filed on Jun. 22, 2020, Korean PatentApplication No. 10-2020-0075915 filed on Jun. 22, 2020, and KoreanPatent Application No. 10-2020-0075916 filed on Jun. 22, 2020 the entirecontents of each of which are incorporated herein by reference in theirentirety.

BACKGROUND 1. Field of the Invention

The following example embodiments relate to a technology forcertificating and synchronizing virtual world and physical world.

2. Description of Related Art

In the existing browsing environment, business operators operatingmedium providing services and third party business operators who areexternal business operators record and use information for users byusing their cookie, respectively. For example, the third party businessoperators expose information suitable for users in real-time throughmedium of the operators by using Programmatic Bidding or RTB (real timebidding) and the like based on information obtained through the cookie.

However, in case of virtual space, since all experiences of users aremade in a provider-centric environment of specific virtual space, thereis a problem that it is hard to effectively collect and confirm userexperience information from the third party business operators' point ofview. Accordingly, in the existing technology, there are limitationsthat the third party business operators simply expose generalinformation to unspecified multiple users regardless of the users'activities in the virtual space or provide relevant information to theusers based on information except for the users' activities in thevirtual space such as users' profiles.

PRIOR ART REFERENCE

Korean Patent Publication No. 10-2002-0007892

SUMMARY

Embodiments of the inventive concept may provide a method and system forcertificating and synchronizing when interconnecting between virtualworld and physical world.

Embodiments of the inventive concept may provide a method and system forcertificating a device in space of virtual world.

Embodiments of the inventive concept may provide a method and system forcertificating space to be register in DTS (Digital Twin Space).

Embodiments of the inventive concept may provide a method and system forperforming user check-in in certified space.

At least one example embodiment provides a computer device configured tocomprise at least one processor implemented to execute acomputer-readable instruction, and by the at least one processor, manageinterconnection relation between virtual space and physical space basedon information of the virtual space and information of the physicalspace, and register a device or space existing in the physical space ina digital object form of the virtual space.

According to an aspect of at least one example embodiment, by the atleast one processor, it may provide a digital twin service through aconnection between an offline device or an offline store and the digitalobject of the virtual space.

According to another aspect of at least one example embodiment, by theat least one processor, it may synchronize user information and activityinformation between the virtual space and the physical space dependingon whether synchronization of a user is permitted or not.

According to another aspect of at least one example embodiment, by theat least one processor, it may connect event information related to thedigital object of the virtual space as the information of the virtualspace.

According to another aspect of at least one example embodiment, by theat least one processor, it may connect information related to an agentof the virtual space as the information of the virtual space.

According to another aspect of at least one example embodiment, by theat least one processor, it may connect information related to currencyof the virtual space as the information of the virtual space.

According to another aspect of at least one example embodiment, by theat least one processor, it may register a device recognizable in thevirtual space by using a code issued through device information orprecertification.

According to another aspect of at least one example embodiment, by theat least one processor, it may register space recognizable in thevirtual space by using map-based data or space information.

According to another aspect of at least one example embodiment, by theat least one processor, it may register software in the digital objectform of the virtual space by using a unique key value.

According to another aspect of at least one example embodiment, by theat least one processor, it may register a group consisting of aplurality of agents in the digital object form of the virtual space.

According to another aspect of at least one example embodiment, by theat least one processor, it may register a device in the form ofprecertification at the time of device production in case of the deviceproviding a connection service with the virtual space.

According to another aspect of at least one example embodiment, by theat least one processor, it may perform a status check of the devicebased on data periodically received from the device or informationcollected in the virtual space.

According to another aspect of at least one example embodiment, by theat least one processor, it may manage information of the physical spacebound in the virtual space, and manage user entry and visit recordsconnected with the space.

According to another aspect of at least one example embodiment, by theat least one processor, it may provide a connection with a networkservice for traffic transmitted from the physical space.

According to another aspect of at least one example embodiment, by theat least one processor, it may buffer or queue data by receiving thedata transmitted from the device or the space, and route the data.

According to another aspect of at least one example embodiment, by theat least one processor, it may manage local information and ownerinformation related to the space as the information of the physicalspace.

According to another aspect of at least one example embodiment, by theat least one processor, it may issue and manage a key used for aconnection service between the virtual space and the physical space.

According to another aspect of at least one example embodiment, by theat least one processor, it may manage permission by each individualservice subject participating in the connection service between thevirtual space and the physical space.

According to various example embodiments, interconnection relationbetween virtual world and physical world may be certificated andsynchronized.

According to various example embodiments, a device may be certificatedin space of virtual world for interconnection between virtual world andphysical world.

According to various example embodiments, space to be registered in DTSmay be certificated for interconnection between virtual world andphysical world.

According to various example embodiments, user check-in may be performedin certificated space.

These and/or other aspects, features, and advantages of the presentdisclosure will become apparent and more readily appreciated from thefollowing description of embodiments, taken in conjunction with theaccompanying drawings of which:

FIG. 1 is a drawing illustrating an example of a network environmentaccording to one example embodiment;

FIG. 2 is a block diagram illustrating an example of a computer deviceaccording to one example embodiment;

FIG. 3 is a drawing for describing spaces according to one exampleembodiment;

FIG. 4 is a drawing illustrating an example of movement between spacesof virtual world according to one example embodiment;

FIG. 5 is a drawing illustrating relation between components composingspace of virtual world according to one example embodiment;

FIG. 6 is a drawing illustrating an example of a digital objectconfigured in space of virtual world according to one exampleembodiment;

FIG. 7 is a drawing illustrating an example of information fordetermining whether to expose digital objects to an agent in space ofvirtual world according to one example embodiment;

FIG. 8 is a drawing illustrating overview of a virtual space compositionsystem according to one example embodiment;

FIG. 9 is a block diagram illustrating an example of internalconfiguration of a virtual space management system according to oneexample embodiment;

FIGS. 10 and 11 illustrate an example of connecting PS (Physical Space)and VS (Virtual Space) according to one example embodiment;

FIG. 12 illustrates an example of a service target based on a connectionfunction between PS (Physical Space) and VS (Virtual Space) according toone example embodiment;

FIG. 13 is a drawing for describing examples of VS (Virtual Space)according to one example embodiment;

FIG. 14 is a block diagram illustrating an example of a certificationand synchronization system for connection between PS (Physical Space)and VS (Virtual Space) according to one example embodiment;

FIG. 15 is a flowchart illustrating an example of a device certificationprocess according to one example embodiment;

FIG. 16 is a flowchart illustrating an example of an agent registrationprocess according to one example embodiment;

FIG. 17 is a flowchart illustrating an example of a space certificationprocess according to one example embodiment;

FIG. 18 illustrates an example of a user check-in process incertificated space according to one example embodiment;

FIG. 19 is a flowchart illustrating an example of a user check-inprocess in certificated space according to one example embodiment;

FIG. 20 illustrates an example of an interface screen for permittingsynchronization when connecting between PS and VS according to oneexample embodiment.

DETAILED DESCRIPTION

Hereinafter, some example embodiments will be described in detail withreference to the accompanying drawings.

FIG. 1 is a drawing illustrating an example of a network environmentaccording to one example embodiment. The network environment of FIG. 1represents an example including a plurality of electronic devices 110,120, 130, 140, a plurality of servers 150, 160, and a network 170. FIG.1 is an example for description of invention, and the number ofelectronic devices and servers is not limited as FIG. 1. Also, thenetwork environment of FIG. 1 just describes an example of one amongenvironments applicable to the example embodiments, and the environmentapplicable to the example embodiments is not limited to the networkenvironment of FIG. 1.

The plurality of electronic devices 110, 120, 130, 140 may be fixedterminals which are implemented with a computer device or mobileterminals. As an example of the electronic devices 110, 120, 130, 140,there are a smart phone, a mobile phone, a navigation, a computer, alaptop, a digital broadcasting terminal, a PDA (Personal DigitalAssistants), a PMP (Portable Multimedia Player), a tablet PC, and thelike. As an example, FIG. 1 indicates a form of a smart phone as anexample of the electronic device 110, but in the example embodiments,the electronic device 110 may substantially mean one of various physicalcomputer devices which may communicate with other electronic devices120, 130, 140 and/or the servers 150, 160 through the network 170 byusing a wireless or wired communication method.

The communication method is not limited, and it may include not only acommunication method using a communication network that the network 170may include (e.g., a mobile communication network, wired Internet,wireless Internet, and a broadcasting network) but also a short-rangewireless communication between devices. For example, the network 170 mayinclude any at least one of networks of PAN (personal area network), LAN(local area network), CAN (campus area network), MAN (metropolitan areanetwork), WAN (wide area network), BBN (broadband network), Internet,and the like. Also, the network 170 may include any one or more amongnetwork topologies including bus network, star network, ring network,mesh network, star-bus network, tree or hierarchical network, and thelike, but it is not limited thereto.

Each of the servers 150, 160 may be implemented with a computer deviceor a plurality of computer devices providing an instruction, a code, afile, a content, a service, and the like by communicating with theplurality of electronic devices 110, 120, 130, 140 through the network170. For example, the server 150 may be a system providing services(e.g., service based on virtual space, instant messaging service, gameservice, group call service (or voice conference service), messagingservice, e-mail service, social network service, map service,translation service, financial service, payment service, search service,content providing service, and the like) to the plurality of electronicdevices 110, 120, 130, 140 connected through the network 170.

FIG. 2 is a block diagram illustrating an example of a computer deviceaccording to one example embodiment. Each of above described pluralityof devices 110, 120, 130, 140 or each of servers 150, 160 may beimplemented by a computer device 200 illustrated through FIG. 2.

Such computer device 200 may include a memory 210, a processor 220, acommunication interface 230, and an input/output interface 240, as shownin FIG. 2. The memory 210 may include a permanent mass storage devicesuch as RAM (random access memory), ROM (read only memory) and a discdrive as computer-readable recording medium. Here, the permanent massstorage device such as ROM and disk drive may be included in thecomputer device 200 as a separate permanent storage device distinct fromthe memory 210. Also, the memory 210 may store OS and at least oneprogram code. The software components may be loaded to the memory 210from the computer-readable recording medium separate from the memory210. The separate computer-readable recording medium may includecomputer-readable recording medium such as a floppy drive, a disc, atape, a DVD/CD-ROM drive, a memory card, and the like. In other exampleembodiments, the software components may be loaded to the memory 210through the communication interface 230 in the computer, not through thecomputer-readable recording medium. For example, the software componentsmay be loaded to the memory 210 of the computer device 200 based on acomputer program installed by files received through the network 170.

The processor 220 may be configured to process computer programinstructions by performing basic arithmetic, logic, and input/outputoperation. The instructions may be provided to the processor 220 by thememory 210 or the communication interface 230. For example, theprocessor 220 may be configured to execute the instructions receivedaccording to a program code stored in a recording device such as thememory 210.

The communication interface 230 may provide a function for communicatingthe computer device 200 with other devices through the network 170. Forexample, a request or an instruction, data, a file, and the like thatthe processor 220 of the computer device 200 generates according to aprogram code stored in a recording device such as the memory 210 may betransmitted to other devices through the network 170 according tocontrol of the communication interface 230. Conversely, a signal or aninstruction, data, a file, and the like from other devices may bereceived to the computer device 200 through the communication interface230 of the computer device 200 by going through the network 170. Thesignal or the instruction, data, and the like received through thecommunication interface 230 may be transmitted to the processor 220 orthe memory 210, and the file and the like may be stored in the storagemedium (the above described permanent storage device) that the computerdevice 200 may further include.

The input/output interface 240 may be a means for interfacing with aninput/output device 250. For example, an input device may include adevice such as a microphone, a keyboard, or a mouse, and the like, andan output device may include a device such as a display, a speaker, andthe like. As another example, the input/output interface 240 may be ameans for interfacing with a device in which an input function and anoutput function are integrated into a single function such as a touchscreen. At least one of the input/output device 250 may be configuredwith the computer device 200 as one device. For example, the touchscreen, the microphone, the speaker, and the like may be implemented ina form included in the computer device 200 as a smartphone.

Also, in other example embodiments, the computer device 200 may includemuch less or much more components than the components of FIG. 2.However, there is no need to clearly illustrate most prior artcomponents. For example, the computer device 200 may be implemented toinclude at least part of the described input/output device 250 orfurther include other components such as a transceiver, a database, andthe like.

FIG. 3 is a drawing for describing spaces according to one exampleembodiment. FIG. 3 indicates PS (Physical Space) 310 which is space ofreal world and VS (Virtual Space) 320 which is space of virtual world,MS (Mixed Space) 330, and DTS (Digital Twin Space) 340.

The space of virtual world may be designed by a provider (businessoperator) of the corresponding virtual world, or configured by a user ora third party business operator who is an external operator. Such spaceof virtual world may be configured in the form of the VS 320, MS 330and/or DTS 340 according to features of the space. The VS 320 may bepure digital based space which may be interworked with a CPS (CyberPhysical System), the DTS 340 may be space interworked with the CPS asvirtual space based on real world, and the MS 330 may be space in whichthe VS 320 and the DTS 340 are mixed. In case of the MS 330, it may beprovided in a form in which an agent which is an object of a user adaptsin a real world environment, or it may be provided in a form ofrendering the real world environment in the virtual world.

The space of the virtual world is a concept of basic virtual space inwhich the agent of the user may be active, and there may be policies forthe agent's activities, information use and/or exposure for each space.The user should recognize that each of Privacy & Terms exists accordingto subject of service provision when using a service in space ofspecific virtual world.

The space of virtual world may be generated or used variously accordingto a provider or a user. For this, a separate gate may be provided to aninterface for movement of an agent between spaces of each virtual world.

FIG. 4 is a drawing illustrating an example of movement between spacesof virtual world according to one example embodiment. FIG. 4 indicatesGate #1 430 for movement between VS #1 410 and VS #2 420 which are twovirtual spaces. Here, the ‘gate’ may be a basic interface providingmovement between spaces of virtual world. Such ‘gate’ may controlmovement between space of virtual world and space of non-virtual worldas well as movement between spaces of virtual world. For example, inFIG. 4, it is indicated that an agent of the VS #1 410 may move to amovement object as space of non-virtual world such as web page,streaming channel, streaming contents, game environment, actual offlinevideo call, and the like. The method for separating space of virtualworld may be determined by a provider of space of the correspondingvirtual world.

The agent may mean a user or a program in the space of virtual world.Here, the agent as the program may be a virtual avatar or personaexisting instead of a third party business operator in a form ofartificial intelligence agent. To such agent, physical features of thespace of virtual world in which the corresponding agent is included maybe applied, and service profile set in the corresponding space ofvirtual world may be applied. Also, the agent may have features based oninformation of a physical device used by a user. For example, the agentmay have field of view according to features of a display of a physicaldevice used by a user or control features according to a controller ofthe corresponding physical device.

In addition, a digital object to be described later may be a genericterm for objects providing a mutual interaction function with the agentas a key element that composes world information in the space of virtualworld.

FIG. 5 is a drawing illustrating a relation between components composingspace of virtual world according to one example embodiment, and FIG. 6is a drawing illustrating an example of a digital object configured inspace of virtual world according to one example embodiment.

FIG. 5 indicates that Space of Virtual World 500 such as the VS 320, theMS 330 and/or the DTS 340 above described through FIG. 3 may includeDigital Object 510. At this time, the Digital Object 510 may be formedin OA (Object Area) which is an area that the Space of Virtual World 500includes. Also, the Digital Object 510 may include DO (Display Object)511, IO (Interaction Object) 512, WO (Web Object) 513 and SO (StreamingObject) 514. At this time, each of the DO 511, the IO 512, the WO 513,and the SO 514 may selectively include Display Area for display ofcontents.

FIG. 6 indicates an example of digital objects arranged in Object Areas610 to 640 formed in Space of Virtual World 600. At this time, thedigital objects may include DOs 651 to 654 which are display objects,IOs 661 to 663 which are interaction objects, WO 671 which is a webobject and/or SO 681 which is a streaming object. Dotted arrows 691 and692 shown in FIG. 6 indicate an example of an agent's movable movement.

As the example of FIG. 6, in the space of virtual world according to oneexample embodiment, various forms of digital objects may exist. Each ofdigital objects may be generated around the agent by determinedconditions, and at this time, the digital object may be generated in theobject area. When the space of virtual world imitates real world such asDTS (e.g., the DTS 340 of FIG. 3), the location of the object area maybe determined based on determined physical location of the real world.For example, if a specific shopping mall or historic site and the likeof the real world are virtualized, the location of the object area maybe determined to have the same configuration with the actual locationconfiguration.

The DO (Display object) may be an object exposed in a spatial form(structural form) in the space of virtual world, and may provide asimple control function to the agent. The IO (Interactive Object) may beexposed in a spatial form in the space of virtual world, and may be anobject having an interactive feature which may interact with the agent.For example, the IO may provide various interaction features with theagent. Also, the WO (Web Object) may be an object in a form of exposingcontents on the web through the space of virtual world. Also, the SO(Streaming Object) may be an object in a form that streaming contentssuch as video, audio, and the like may be continuously exposed.

The OA may have a feature of the space capable of generating a digitalobject which is a virtual object in the space of virtual world. Thedigital object existing in the OA may basically have the followingfeatures. Each function may be determined for whether it is provided ornot based on a contract between an owner of the digital object and aprovider of the space of virtual world.

1. Functions related to agent interwork

1) Interaction function: a function for providing an interface which maybe directly controlled by an agent

2) History function: a function for tracking and storing informationrelated to agent's visit and interaction

3) Favorite/Like function: a function for the agent to storefavorite/like information for the digital object

4) Follow me function: A function for making the digital object in theOA follow around the agent by making the digital object into smallwindows or icon.

2. Functions related to external API interwork

1) A function capable of sharing information for generation time and endtime of the digital object in the space of virtual world

2) An interface function for providing information related to the agentcollected by the digital objects in the OA to the outside

3) An interface function for reflecting real world information andexpressing it in the DO or the IO (e.g., a function for expressinginformation such as order waiting, congestion in store, parkingsituation, and the like of actual store)

3. A function related to gate interwork

1) Gate connection function: a function for connecting the agent toanother VS or PS in the digital object existing in the OA

4. Functions related to DA (Display Area)

1) DA function: an area, in which an owner of the digital object mayexpose contents according to the owner's intention, may be exposed toagents.

2) Display content request function: a function for receiving externalcontents through Real Time Content Control Module.

Meanwhile, the DO and the IO which are parts of elements consisting ofthe space of virtual world may exist in the OA. The DO and the IO arebasically indicated in the form of being displayed on the user's screen,and may have characteristics of a structure having a certain standard.The physical information of the DO may be provided to the provider ofthe space of virtual world by the owner or may be directly designed byusing templates provided by the provider or using predetermined tools.Also, each of the DO and the IO may have the DA that the owner maydirectly control. The contents exposed in the DA may be providedaccording to selection of the owner of the DO and/or the IO withinproper range based on policies between the owner of the DO and/or the IOand the provider of the space. The DA may individually have exposurefeatures (features for perspective, or ignoring and emphasizingperspective, and the like) according to field of view of the agent andinteraction features. In addition, each IO may have interactive featuresthat the owner of the IO may directly control. The JO may provideinteraction suitable for the agent's situation based on the interactionfeature activation information of the agent. For example, when the userwears only HMD (Head Mounted Display), the IO may provide an operationfunction which is the same with the DO. On the other hand, when the userhas a controller combined with the HDM, interaction features (e.g.,grabbing, pulling, drawing, and the like) with the agent provided in thespace of the corresponding virtual space may be provided. As anotherexample, when the user has an additional motion recognition controllerand the like, the corresponding features may be used. In case of the DO,it may have basic control features. Such basic control features mayinclude a basic control function of the space of virtual world set bythe provider. For example, the basic control function may includehistory set and/or favorite/like information set and the like.

FIG. 7 is a drawing illustrating an example of information fordetermining whether to expose digital objects to an agent in space ofvirtual world according to one example embodiment. FIG. 7 shows ObjectArea 710, Digital Object 720, and Agent 730 arranged in Space of VirtualWorld 700. The Space of Virtual World 700 may be interworked withPhysical World (real world) 740. Here, features of the Digital Object720 (Digital Object Features) may include features of the Object Area710 (Object Area Features). The features of the Object Area 710 mayinclude basic information for a specific area in the Space of VirtualWorld 700. For example, multiple sub-digital objects may exist in theSpace of Virtual World 700, and each sub-digital object may follow theexisting policies of the corresponding object area. Also, as alreadydescribed, the Digital Object 720 may be one of DO, IO, WO, and SO.Therefore, the features of the Digital Object 720 may include one ofDO's features, IO's features, WO's features, and SO's features. Inaddition, the features of the Digital Object 720 may further includefeatures of Display Area 750 (Display Area Features) that the DigitalObject 720 includes and Tracking Features 760 for the Agent 730.

At this time, to determine whether the Digital Object 720 is exposed tothe Agent 730 in the Space of Virtual World 700, (1) features of theDigital Object 720, (2) FoV (Field of View) of the Digital Object 730,and (3) distance between the Agent 730 and the Display Area 750 may beused.

FIG. 8 is a drawing illustrating overview of a virtual space compositionsystem according to one example embodiment. A Virtual Space CompositionSystem 800 of FIG. 8 may include a VR (Virtual Reality) Client Device810, a Mobile Edge Computing Device 820, an Edge Computing Device 830,and a Cloud Computing System 840. Each of the VR Client Device 810, theMobile Edge Computing device 820, the Edge Computing Device 830, and theCloud Computing System 840 may be implemented through the computerdevice 200 described above through FIG. 2. It may be easily understoodthat the Cloud Computing System 840 may be implemented by two or morecomputer devices 200.

The VR Client Device 810 may be a physical device for displaying thespace of virtual world to a user, and as shown in FIG. 8, it may includea Rendering Machine 811, a VSC (Virtual Space Controller) 812 that theRendering Machine 811 may include, a HMD (Head Mount Display) 813, andControllers & Sensors 814. The Rendering Machine 811 may display thespace of virtual world in the HMD 813 through the VSC 812, and outputvalue of the Controllers & Sensors 814 may be used to control the user'sagent in the space of virtual world and/or interact with a digitalobject located in the space of virtual world.

According to example embodiments, the VSC 812 may not be directlyincluded by the VR Client Device 810 or the Rendering Machine 811, andmay be included in the Mobile Edge Computing Device 820 or the EdgeComputing Device 830. For example, when the user's VR Client Device 810is a device capable of directly rendering the space of virtual world,the space of virtual world may be rendered by using the VSC 812 of theVR Client Device 810. On the other hand, when the user's VR ClientDevice 810 may not directly render the space of virtual world, the VRClient Device 810 may render the space of virtual world through a VSC821 of the Mobile Edge Computing Device 820 wirelessly connected throughan Access Point 822 or a VSC 831 of the Edge Computing Device 830wired-connected and display it in the HDM 813. When the Mobile EdgeComputing Device 820 and the Edge Computing Device 830 are notsupported, as described later, the space of virtual world may berendered by using a Virtual Space Controller farm 842 included by theCloud Computing System 840. The Virtual Space Controller farm 842 maygenerate instance of VSC for the user and support the user to displaythe rendered space of virtual world in the HMD 813. Here, the VSCs 812,821, and 831 may be VSC instances generated and provided by the CloudComputing System 840 through the Virtual Space Controller farm 842.

The VSCs 812, 821, and 831 may render the space of virtual world so thatcontents transmitted in relation to the display area of the digitalobject may be displayed in the space of virtual world for the agent. TheVSCs 812, 821, and 831 may be generated for the user's VR Client Device810 corresponding to the agent by the Cloud Computing System 840configuring the space of virtual world and providing the service, andlaunched on at least one of the VR Client Device 810, the Mobile EdgeComputing Device 820, the Edge Computing Device 830 or the CloudComputing System 840, and may support rendering of the space of virtualworld for the VR Client Device 810.

Meanwhile, the VR Client Device 810 may receive a service related to thespace of virtual world by being directly connected to the CloudComputing System 840 or connected to the Cloud Computing System 840through the Mobile Edge Computing Device 820 or the Edge ComputingDevice 830.

Such Cloud Computing System 840 may be an operator's system forconfiguring the space of virtual world and providing a service relatedto the space of virtual world to the user. Such Cloud Computing System840 may include a Virtual Space Management System 841 and the VirtualSpace Controller farm 842 as shown in FIG. 8. The Virtual SpaceController farm 842 may be implemented in a form of being included inthe Virtual Space Management System 841. The Virtual Space ManagementSystem 841 will be described in detail later through FIG. 9.

FIG. 9 is a block diagram illustrating an example of internalconfiguration of a virtual space management system according to oneexample embodiment. The Virtual Space Management System 841 according toexample embodiments may include a Client I/F 910, an Agent TrackingManagement Module 920, an Agent Information Management Module 930, aVirtual World Information Management Module 940, a Real Time contentControl Module 950, a Physical World Interface Module 960, and theVirtual Space Controller farm 842 as shown in FIG. 9. Such components ofthe Virtual Space Management System 841 may be functional expressions ofthe processor 220 included by the at least one computer device 200implementing the Cloud Computing System 840.

The Client I/F 910 may provide a user interface for the VR Client Device810. For example, the Client I/F 910 may provide various user interfacesthat the user may interact with the space of virtual world in a processthat the user receive a service for the space of virtual world providedby the Cloud Computing System 840 by using the VR Client Device 810.

The Agent Tracking Management Module 920 may track an agent located andmoving in the space of virtual world. The information obtained bytracking the agent may be stored in real time in relation to anidentifier of the corresponding agent in Tracking DB, and the trackinghistory information for the agent may be stored in relation to theidentifier of the corresponding agent in Tracking History DB.

The Agent Information Management Module 930 may store the agent'sprofile and the agent's consent. For example, the Agent InformationManagement Module 930 may store the agent's profile in relation to theidentifier of the corresponding agent in Agent Profile DB, and may storethe agent's consent in relation to the identifier of the correspondingagent in Agent Consent DB. Here, the consent may include consent forPrivacy & Terms.

The Virtual World Information Management Module 940 may manageinformation for the space of virtual world. For example, the VirtualWorld Information Management Module 940 may store information forpolicies of the space of virtual world, Virtual Map Information,location information of the space of virtual world (e.g., GPSinformation for the space of virtual world), information of the digitalobject located in the space of virtual world, and may providecorresponding information according to request from other modules.

The Real Time Content Control Module 950 may select content displayed inthe space of virtual world. For example, the Real Time Content ControlModule 950 may select content to be displayed through display area ofthe digital object configured in the space of virtual world. For this,the Real Time Content Control Module 950 may include a function forcontent bidding and a function for selecting content to be displayed.For example, the Real Time Content control Module 950 may select contentto be displayed through display area based on bidding for a Third PartyDisplay Content Provider 970.

The Physical World Interface Module 960 may provide a function forcontrolling physical resource and an interface for physical resource.

The Virtual Space Controller farm 842 may generate, provide, and manageinstance of the VSC to help the rendering of the VR Client Device 810 asdescribed above.

Such Virtual Space Management System 841 may receive HMD information andcontrol/sensor information from the VR Client Device 810. The HMDinformation may include 3-DoF (Degree of Freedom) or 6-DoF's devicemotion tracking information. Here, the device motion trackinginformation may include motion tracking information for the VR ClientDevice 810. Also, the HMD information may include user information fromthe VR Client Device 810. For example, the user information may includerotation of the user's head, movement of the user's body, and eye-gazeinformation of the user. Also, the HMD information may includeinformation relating to rendering such as rendering delay, temperature,and the like. Meanwhile, the control/sensor information may includeinformation for movement of the user's body or real time controllerinformation such as key events, movement and the like.

Hereinafter, specific embodiments of a method and system for providing aservice through connection between a user, a device, and a space in VRspace will be described.

In order to provide a service based on a connection function betweenvirtual world and real world, trust between the digital object and theagent is required.

(1) Example 1: An Agent Service May be Differentiated Through ConnectionBetween an Offline Store and a Digital Object

When the digital object proceeds with an event targeted at the agent, adifferentiated service is provided in the VS if a user has a history ofvisiting an offline store and purchasing in a situation that a realworld store is in the form of digital object in the VSCs 812, 821, 831.

When the digital object tries to proceed with a personalized service,event, and the like by using the agent information as an individualservice subject, a method for providing data based on trust between thedigital object and the agent is required.

Also, from the point of view of the digital object, a means forconfirming whether the collecting agent's profile is reliable should beprovided.

(2) Example 2: An Interconnection Service Between an Offline Device anda Virtual Device in VS May be Provided

FIG. 10 shows an example of connecting PS 1010 corresponding to anoffline fitness center and a device in the PS 1010 with VS 1020 and adigital object in the VS 1020.

As a user visits the fitness center in the PS 1010 which is the realworld and exercises on a device certificated in the form of DigitalTwin, the agent of the VS 1020 also visits the fitness center in the VS1020 and exercises, and so changes in the agent's shape may occur andrewards may be made accordingly.

When the user exercises in real offline, if the PS 1010 fitness centerprovides a digital twin function with the VS 1020, and if the agent ofthe user wants to show simultaneity in cyber space, the correspondingagent may imitate the exercising through the corresponding device in theVS 1020.

Or, in the fitness center side, in order to show how active offlinesubscribers in the VS 1020 are, a random agent provided by a serviceprovider, not the agent of the actual user, may shows the exercising. Ageneral agent visiting the fitness in the VS 1020 may determine thedegree of the activation of the corresponding offline environment byconfirming virtual information currently used by the digital twin-basedagent of the registered user. Through this, unsubscribed agent visitingthe fitness center in the VS 1020 may determine whether to subscribe inthe real world by checking the activated level of the correspondingfitness.

In addition, exercise history may be reflected in real time or may bereflected by the user's decision in the VS 1020 according to the user'sselection. In this process, the fitness center operator may select realtime or non real time interwork according to the digital twin'ssubscription model.

(3) Example 3: When Proceeding with Dating in the VS, a Method forCertificating the Corresponding Agent's Identity is Required

FIG. 11 shows an example of a shopping mall providing a digital twinservice simulating PS 1110 which is real world. Stores in the shoppingmall in the PS 1110 may be represented as digital objects in VS 1120based on status information of individual store. In the VS 1120, avirtual user generated based on the number of users entering and leavingthe shopping mall may be included, and when it is possible to understandlocation such as a proximity sensor, user processing is possible basedon location information. In the VS 1120, an agent that wants to expresshis or her information among users who checked in a specific store, oran agent generated as a virtual customer in the VS 1120 based oninformation checked in a specific store may be included, and at thistime, the difference between the agents may be represented byclassifying user features such as face, clothes, condition, and the likein image so that they may be visually distinguished. When a user whochecked in a specific store allows his or her agent to be exposed in theVS 1120, the corresponding agent may have explicit surface includinguser information unlike a ghost agent.

FIG. 12 illustrates an example of a service target based on a connectionfunction between PS and VS. In this embodiment, a method for mutualconnecting and certificating between the PS and the VS is required.Providing a system for organizing information capable of interworking ineach area and interworking it is required. At this time, information formutual connecting between the PS and the VS may be exposed or controlledaccording to the situation, and the corresponding information may besupplied to the desired source of demand.

Referring to FIG. 12, a Certification Controller 1210 provides a keyfunction for managing mutual connection in connecting the VS and PS.Basically, the Certification Controller 1210 provides a registrationprocess for connecting information in the VS and the PS, and providescertification for whether the space exists in real world. Theinformation for mutual connection between the PS and the VS may becontrolled by a VSC 1220 (corresponding to one of 812, 821, 831), andmay be connected with a reliable real world data collecting system suchas a Data Creator 1230 and share data with a Certified Data Consumer1240 as an organization using it.

In a Digital Element with Physical 1250, other information (e.g. checkin information of shopping mall or proximity sensor information, trafficjam, and the like) connected with space information such as informationrelated to environment (e.g. air, wind, temperature, and the like) in VSenvironment is included. The Digital Element with Physical 1250 is ageneric term for other entities, not entities with a separate owner suchas a Digital Object 1260 (DO, IO, and the like), and may includeinformation connected with the PS.

The Data Creator 1230 may include information which may be reflected inthe VS as external API creating actual user information, and forexample, a certificated fitness application and the like may correspondto this.

FIG. 13 shows examples of VS. In VS 1310, each space has a spatialspecificity similar to that of actual PS when providing a service basedon digital twin.

A Public Space 1311 which is one of the VS 1310 means a space madepublic for an agent of the VS 1310 such as a square, a park, and thelike, and a separate agent permission is not required to enter thePublic Space 1311, and real world devices connected to the Public Space1311 are controlled by a VSC (corresponding to one of 812, 821, and831).

A Commercial Space 1312 which is another of the VS 1310 means a digitalobject paid by a specific business operator such as a store, andgenerally means a space simulating an offline store in real world. TheCommercial Space 1312 basically allows an anonymous agent to enter, butaccording to characteristics of the corresponding space owner, onlyregistered agent may access to space of some digital object. The deviceconnected with the Commercial Space 1312 may be interworked through theVSC (corresponding to one of 812, 821, and 831), and the space owner maybe authorized control rights for the Commercial Space 1312 from the VSC.

A Home (Personal) Space 1313 which is another of the VS 1310 meansprivate space as digital object configured by an individual. For theHome (Personal) Space 1313, basically only owner of the correspondingspace may be access, and registered agent may be access according torequest of the corresponding space owner. The device connected with theHome (Personal) Space 1313 may be interworked through the VSC(corresponding to one of 812, 821, and 831), and the space owner may beauthorized control rights for the Home (Personal) Space 1313 from theVSC.

In case of the agent using the above described VS 1310 and digital twinservice, VS information owned by the agent and information of theinterworking device in the VS may be exposed based on certification. TheVSC (corresponding to one of 812, 821, and 831) may use the VSinformation and device information in unidentified or anonymized form.

When the agent visits a specific store corresponding to the CommercialSpace 1312 in the VS 1310 by using the VS information and the deviceinformation, the corresponding store may recommend customized productsbased on the user information in the VS 1310. For example, if there is acoffee machine in the Commercial Space 1312, it may recommend the user'sfavorite coffee beans, and if there is an audiobook, it may recommend anaudiobook of a similar genre that the user prefers.

A certified space means a space that simulates a space existing in realworld (e.g. store, brand, and the like) in the VS 1310 and provides afunction mutual connected with real world. Requirements of the certifiedspace may include (1) user check-in, (2) connection with POS (point ofsales) for membership, purchase, loyalty, and the like, (3) connectiongwith a device certified as a device providing a specific service, andthe like. When using a certified device such as store BGM, fitness GYM,and the like in the certified space, additional rewards may be provided.

FIG. 14 is a drawing illustrating an overview of a Certification andSynchronous System according to one example embodiment. A Certificationand Synchronous System 1400 of FIG. 14 provides a process forcertification and synchronization (interwork) for connection between PS1401 and VS 1402.

The Certification and Synchronous System 1400 may be implemented throughthe computer device 200 described through FIG. 2. According to exampleembodiments, the Certification and Synchronous System 1400 may beimplemented in the form of being included in the Virtual SpaceManagement System 841 described through FIG. 8 or may be implemented inthe form capable of interworking with the Virtual Space ManagementSystem 841 as a separate system.

The Certification and Synchronous System 1400 may include the servicetarget (the Certification Controller 1210, the VSC 1220, the DataCreator 1230, and the like) described through FIG. 12.

Referring to FIG. 14, the Certification and Synchronous System 1400 mayinclude a Virtual Space Connector 1410, a Registration Control 1420, aCommon Control 1430, a Space Control 1440, and a Physical SpaceConnector 1450.

First, the Virtual Space Connector 1410 plays a role providing aconnection function between services provided in the Certification andSynchronous System 1400, and may include an Object Connector 1411, AgentActivity Connector 1412, and a Cyber Currency Connector 1413.

The Object Connector 1411 may connect event information related todigital objects (DO, 10, and the like) on VS. For example, the ObjectConnector 1411 may provide information of on/offline advertisementconnection and event, and the like, and at this time, video stream inreal environment may be used.

The Agent Activity Connector 1412 may connect information related to theagent on VS. A service in the form of virtually exposing the agentinformation of the user who checked in offline is possible online. Forexample, the Agent Activity Connector 1412 may expose the correspondingmotion on virtual agent of VS when the user actual exercises in realworld fitness GYM and the like, and visual effect according to theamount of exercise may be reflected to the agent in VS.

The Cyber Currency Connector 1413 may connect information related tocurrency in VS. For example, the Cyber Currency Connector 1413 mayinterwork data received from a store POS and interwork point and thelike in VS.

In addition, the Registration Control 1420 may include aPre-Certification Manager 1421, a Health Check Manager 1422, and aRegistration Manager 1423.

The Pre-Certification Manager 1421 may issue a code (e.g. OTP and thelike) that may be preregistered at the time of device production or atthe time of production when an API function in software is built-in. ThePre-Certification Manager 1421 may proceed with registration in the formof pre-certification when a digital twin device production company or asoftware company mass produces certified devices providing a connectionservice with VS. For example, a check-in device, a POS device, a WaitingQueue Module, and the like correspond to it, and hardware or software isunrelated.

The Health Check Manager 1422 may take charge of status check of theregistered device. The Health Check Manager 1422 may periodicallyreceive data from the device or may operate status check in a method forcollecting information by request from VS.

The Registration Manager 1423 may provide a function for individualregistration of device and software. In case of the device, theRegistration Manager 1423 may register by using device information(e.g., UUID, unique IME, MAC address, and the like) or using aregistration code based on OTP issued by using the Pre-CertificationManager 1421. In case of software, the Registration Manager 1423 maycertify by using a unique key value, and may provide a separatecertification module on VS. In case of the space, the RegistrationManager 1423 may use map based data (e.g. identifiable information suchas land registration data, road view data, satellite photograph, and thelike) or a separate certification method (e.g., franchise information,land register, and the like). At this time, the Registration Manager1423 may proceed with space registration by interworking spaceinformation provided by a third party business operator. In case of agroup, the Registration Manager 1423 may have information of agentregistered with a specific number of people or more, and may give whenpaying costs for owning the corresponding group name on VS. Thecertified group means a group to use the same environment (rights) withreal world in the VS 1402 through certification in the VS 1402 among aspecific interest group and a real world group, and for example, socialservice page form may be spread to the group. It may set conditions toform a group based on certificated information in real world or the VS1420, not based on naming and the like, and at this time, the conditionsmay apply a rule that there are N or more certification agents in the VS1402. Or, when paying a certain cost required by the VS 1402, a groupname in the corresponding VS 1402 may be uniquely given like a domain.Through this, the Registration Manager 1423 may differentiate from anagent that does not meet the corresponding conditions by forming a groupmeeting the conditions as certificated information in the VS 1402.

In addition, the Common Control 1430 may include a Key Manager 1431, aPermission Manager 1432, and a Security Manager 1433.

The Key Manager 1431 may provide a function for issuing and managing akey used for a service throughout the service.

The Permission Manager 1433 may provide a function for managingpermission by each individual service subject (e.g. agent, device, spaceowner of digital objects, group owner).

The Security Manager 1433 may provide a security function for eachmodule according to interworking between the PS 1401 and the VS 1402.

Also, the Space Control 1440 provides a function for managinginformation of the PS 1401 bound to the VS 1402, and may include a POSManger 1441, a Marketing Manager 1442, and a Check-in Manager 1443.

The POS manager 1441 plays role managing POS sales information. The POSManger 1441 may manage rewards information such as points, coupons, andthe like in addition to sales information.

The Marketing Manager 1442 may be used when proceeding with theinterworked event between the PS 1401 and the VS 1402, and for example,it may update online information when checking in offline.

The Check-in Manager 1443 may provide a management function for userentering and visiting history connected with space (PS 1401 and VS1402).

Last, the Physical Space Connector 1450 provides a function forconnecting information in the PS 1401 which is real world. Theconnection object may include a device type as well as a software type.The Physical Space Connector 1450 may include a Network Adaptor 1451, aStream Manager 1452, a Location Information Provide 1453, and a Check-inModule 1454.

The Network Adaptor 1451 may connect a network service for traffictransmitted from the PS 1401. The Network Adapter 1451 may supportvarious network protocols (e.g., TCP, HTTP, and the like) forcompatibility for each device or each service.

The Stream Manager 1452 plays a role processing data consistentlyreceived from a device or the PS 1401. The Stream Manger 1452 mayprovide a function for receiving large scale stream and buffering orqueuing it, and then, may provide a function for routing the receivedstream.

The Location Information Provider 1453 may manage connection informationwith service business operator supplying local information. The LocationInformation Provider 1453 may mange main information such as real estateor downtown information for space management. Business operatorsproviding space owner information may correspond to the managementobject.

The Check-in Module 1454 may manage user entering and visiting historyconnected with the space through interworking with the Check-in Manager1443.

FIG. 15 is a flowchart illustrating an example of a device certificationprocess according to one example embodiment. A device certificationprocess according to example embodiments may be performed by thecomputer device 200 implementing the Certification and SynchronousSystem 1400.

In Step 1510, the Certification and Synchronous System 1400 may enter adevice registration process in accordance with the device owner'srequest and proceed with the corresponding process.

In Step 1520, the Certification and Synchronous System 1400 may setdevice information to be registered as an individual registrationfunction for a device. At this time, the device information may includebasic information for recognizing the corresponding device in the VS.When providing a service by connected in the VS, permission related todata access between the owner and the VS should be mutually set, and theVS service provider should notify to an agent using the correspondingdevice. The Certification and Synchronous System 1400 may set devicename, device ID (serial number or unique key), device data interface(e.g., API and the like), tracking permission information, device UI(e.g. 3D rendering information) and the like as the device information.In case of the device ID, when a device producer provides a uniqueregistration function to the VS service business operator, it needs tobe registered in advance. For the tracking permission information, amongthe data acquired by the VS, the device owner and the agent may providewhether the data is opened or not in the permission form.

In Step 1530, the Certification and Synchronous System 1400 may setspace connection information for the device registered in Step 1520. Thespace connection information may be registered by the owner of thedigital object through a contact with the VS service provider orindividually registered by individual users. The owner of the digitalobject may provide a service based on connection between spaces in theVS or connection with several offline branches. At this time, the VSservice provider may additionally request costs for registration, andmay set limitation for registration. The Certification and SynchronousSystem 1400 may set the number of connectable devices per space, thenumber of connectable agents per device, the number of devices that maybe updated simultaneously, the number of device that may be exposedsimultaneously, the number of devices that may be controlledsimultaneously, and the like as the space connection information.

In Step 1540, the Certification and Synchronous System 1400 may setagent access rights information for the device registered in Step 1520.The Certification and Synchronous System 1400 may authorize each agentthat may interwork with the device connected with the VS with accessrights for the corresponding device.

In Step 1550, the Certification and Synchronous System 1400 may reviewand verify a response for the VS in which the device information and thespace connection information are set. The Certification and SynchronousSystem 1400 may check status of the device by a method for periodicallyreceiving data from the registered device or collecting information byrequest from the VS.

In Step 1560, the Certification and Synchronous System 1400 may confirmthe final review for the corresponding device from the register who isthe device owner.

In Step 1570, the Certification and Synchronous System 1400 may providea service based on digital twin through connection between the device inthe VS and the agent if the device registration is completed.

FIG. 16 is a flowchart illustrating an example of an agent registrationprocess according to one example embodiment. The agent registrationprocess according to example embodiments may be performed by thecomputer device 200 implementing the Certification and SynchronousSystem 1400.

In Step 1610, the Certification and Synchronous System 1400 may certifyan owner of digital object. The Certification and Synchronous System1400 may perform certification for the owner of t digital objectproviding a service based on connection between spaces in the VS orconnection with several branches.

In Step 1620, the Certification and Synchronous System 1400 may inputagent information to be interworked with the digital object.

In Step 1630, in case of registering an agent corresponding to the inputagent information, the Certification and Synchronous System 1400 maydetermine whether it is out of range of the agent that may beinterworked with the digital object.

In Step 1640, when it is not out of the corresponding range within therange of the agent that may be interworked with the digital object, theCertification and Synchronous System 1400 may complete the correspondingagent registration according to the input agent information.

In Step 1650, when it is out of range of the agent that may beinterworked with the digital object, the Certification and SynchronousSystem 1400 may request additional cost by calculating a probable costfor the digital object, and may confirm payment for the correspondingcost.

In Step 1660, the Certification and Synchronous System 1400 may updatethe range of the agent that may be interworked with the digital objectby inputting the space connection information according to the paymentof the cost.

In Step 1670, the Certification and Synchronous System 1400 may completethe corresponding agent registration as the range of the agent that maybe interworked with the digital object is updated.

The Certification and Synchronous System 1400 may set additional agentinterworking information to the previously registered digital object.

A device register may set access rights of the agent for thecorresponding device. For example, in case of a product seller,registration rights for a corresponding product is acquired through theCertification Controller 1210, and based on this, the maximum number ofthe agents that may be registered may be determined. For example,maximum two agents may be registered for one device code.

After this, in the situation that an actual purchase is made, a user mayproceed with a process for certification of registration code of anactual product and use agent. After initial certification, the owner ofthe corresponding digital object may register an additional user agentwithin the determined range of agent registration available. At thistime, the rights of the owner of the digital object may be authorized.

Separately, when an owner of the Commercial Space 1312 or the Home(Personal) Space 1313 tries to provide a device existing in his or herPS to several agents, separate rights are obtained from theCertification Controller 1210.

FIG. 17 is a flowchart illustrating an example of a space certificationprocess according to one example embodiment. The space certificationprocess according to example embodiments may be performed by thecomputer device 200 implementing the Certification and SynchronousSystem 1400.

The Certification and Synchronous System 1400 may register space forproviding a service based on digital twin in DTS.

In Step 1710, the Certification and Synchronous System 1400 may selectOA (object Area) or digital object (DO, IO, and the like) in DTS forspace registration. In order for a real world owner to search the OA ordigital object in DTS, an interface that may be searched by inputtingaddress, phone number, and the like (when information in the VS isconnected with map data) or an interface that may be selected in theform of directly pin it by using a map and the like.

In Step 1720, the Certification and Synchronous System 1400 may performowner certification of PS corresponding to the OA or the digital objectselected in Step 1710. For owner certification of an actual space, a VSmanager may use mutual information in a map and the like to usemap-based data. When a specific owner such as a franchise and the liketries to register mapping information, an interface that may directlyinput relevant information by using OCR and the like may be provided.

In Step 1730, the Certification and Synchronous System 1400 may set aservice range of space certified through the above described process1710, 1720, i.e., a range of an actual space. For example, theCertification and Synchronous System 1400 may set a service range byusing land registration range of map data. As another example, theCertification and Synchronous System 1400 may also set a service rangethrough a selection method by using actual modeling information. Aftersynthesizing images for space by using a scanning by using a drone andthe like or a depth sensor and the like, rights demand for thecorresponding space may be applied. As another example, theCertification and Synchronous System 1400 may apply differentiateddemand according to the space set range based on differentiated gradeaccording to registration cost, advertisement exposure space, and thelike.

In Step 1740, the Certification and Synchronous System 1400 may proceedwith certification of a device to be interworked with the certifiedspace. When the Certification and Synchronous System 1400 tries toregister a device in the interworking space, additional registration forthe device may be proceeded.

In Step 1750, the Certification and Synchronous System 1400 may setagent viewing and tracking permission for the certified space. TheCertification and Synchronous System 1400 may set agent information andthe like to acquire in the interworking space as information between theinterworking space and the agent, and also may request agent trackinginformation within the interworking space according to the cost.

FIG. 18 illustrates an example of a user check-in process incertificated space according to one example embodiment. The usercheck-in process according to example embodiments may be performed bythe computer device 200 implementing the Certification and SynchronousSystem 1400.

Process 1) is that after arriving in PS 1810 corresponding to acertified space, a user proceeds with check-in by using his or hermobile device. As a check-in method, a method for processing check-in byusing a signal which may be acquired from the mobile device, e.g., GPS,WiFi, beacon, and the like, a method for processing check-in by using QRcode, a method for processing check-in based on AR based spacerecognition technology, and the like may be used.

Process 2) is that the user may selectively determine whether tosynchronize check-in between the certified space and VS 1820 forinterworking with an agent in the VS 1820.

Process 3) is that agent entry in the VS 1820 may be permitted. When theagent connects to a specific space range in the VS 1820, the agent mayenter the VS 1820 of the corresponding space after usercertification/noncertification.

The Certification and Synchronous System 1400 may generate and provideonetime check-in code for check-in of the user. The user may connect tothe certified space through the onetime check-in code by using themobile device when checking-in. The check-in information of the user forthe certified space is recorded in the VS 1820, and when using thedevice registered in the certified space, the use information may bealso recorded.

A differentiated service may be provided according to a check-in methodbased on macro such as GPS and a check-in method such as WiFi or beaconand the like on offline according to owner selection of digital objectin the VS 1820.

FIG. 19 is a flowchart illustrating an example of a user check-inprocess in certificated space according to one example embodiment. Theuser check-in process may be performed by the computer device 200implementing the Certification and Synchronous System 1400.

In Step 1910, the Certification and Synchronous System 1400 may receiveuser check-in request in PS from a user device.

In Step 1920, the Certification and Synchronous System 1400 may receiveselection for a check-in method from a user for the user check-inrequest. The user check-in method may include a check-in method usingonetime code such as QR code, barcode, pin number, and the like, acheck-in method using location based certification such as GPS, WiFi,beacon, and the like, a check-in method through AR based space orsurface (e.g. object or sculpture and the like for certifying a specificstore) recognition, as described above.

In Step 1930, when the check-in method using a separate onetime code isselected in Step 1920, the Certification and Synchronous System 1400 mayrequest a certification code through the Check-in Manager 1443. At thistime, the certification code request should be provided with PSinformation, and the corresponding information may composed of locationbase, registered SSID (Service Set Identifier) base, beacon ID base, QRcode or barcode base or a separate pin number, and the like, and ARbased surface detection point information of certification recognitionobject and the like may be included.

In Step 1940, the Certification and Synchronous System 1400 may confirmwhether to register for PS that requested user check-in through theRegistration Manager 1423 based on PS information included in thecertification code request. The Certification and Synchronous System1400 may confirm whether to register for the PS that the user tries tocheck in by using map based data (e.g. identifiable information such asland registration data, road view data, satellite photograph, and thelike) or a separate certification method (e.g. franchise information,land register, and the like).

In Step 1950, when the PS that requested the user check-in correspondsto the space registered in advance, the Certification and SynchronousSystem 1400 may issue a certification code for user check-in.

In Step 1960, the Certification and Synchronous System 1400 mayrecognize code connection through the user device or a separate device,i.e., connection of the certification code issued in Step 1950.

In Step 1970, the Certification and Synchronous System 1400 may confirmuser check-in based on the user information according to code connectionor device information. When the check-in method selected in Step 1920does not use a separate onetime code, the code generation process isskipped, and the user check-in may be proceeded by directly enteringStep 1970.

In Step 1980, the Certification and Synchronous System 1400 may updateagent information in the VS according to the user check-in and mayinterwork check-in information in real time. The check-in informationmay be exposed or not exposed to different digital object or agentaccording to user setting. For example, even if check-in is performed inreal world, the corresponding check-in information is not updated to theuser agent of the VS based on the setting value for the agent in the VS.

The Certification and Synchronous System 1400 may provide a permissioncontrol function of the agent when connecting the PS and the VS. Whenconnecting user information and activity information in the PS and theVS, whether to synchronize the corresponding information, whether toexpose (reflect) and the like may be determined according to selectionof the user. Referring to FIG. 20, the Certification and SynchronousSystem 1400 may provide a synchronization object list 2010 forsynchronization permission through a synchronization permission screen2000. A user may selectively set a synchronization object such ascheck-in, payment, activity, display use, and the like through thesynchronization object list 2010, and permit synchronization between thePS and the VS for the selected object. When connecting a registeredoffline store and online store, an owner of a digital object may controlsetting related to synchronization between the corresponding spaces.Control point for synchronization is available in the synchronizationpermissionl screen 2000 provided in a mobile device of the user and acontrol panel in the VS.

Likewise, According to the example embodiments, mutual connectionrelation between virtual world and physical world may be certified andsynchronized, and space certification and device certification may beperformed for mutual connection between virtual world and physicalworld, and according to example embodiments, user check-in in acertified space may be performed.

The units described herein may be implemented using hardware components,software components, and/or a combination thereof. For example, aprocessing device may be implemented using one or more general-purposeor special purpose computers, such as, for example, a processor, acontroller and an ALU (arithmetic logic unit), a digital signalprocessor, a microcomputer, a FPGA (field programmable gate array), aPLU (programmable logic unit), a microprocessor or any other devicecapable of responding to and executing instructions in a defined manner.The processing device may run an operating system (OS) and one or moresoftware applications that run on the OS. The processing device also mayaccess, store, manipulate, process, and create data in response toexecution of the software. For purpose of simplicity, the description ofa processing device is used as singular; however, one skilled in the artwill be appreciated that a processing device may include multipleprocessing elements and multiple types of processing elements. Forexample, a processing device may include multiple processors or aprocessor and a controller. In addition, different processingconfigurations are possible, such as parallel processors.

The software may include a computer program, a piece of code, aninstruction, or some combination thereof, for independently orcollectively instructing or configuring the processing device to operateas desired. Software and data may be embodied in any type of machine,component, physical or virtual equipment, computer storage medium ordevice to provide instructions or data to or be interpreted by theprocessing device. The software also may be distributed over networkcoupled computer systems so that the software is stored and executed ina distributed fashion. In particular, the software and data may bestored by one or more computer readable recording mediums.

The method according to the example embodiments may be implemented in aform of program instruction which may be performed through variouscomputer means and recorded in computer-readable media. The media mayalso include, alone or in combination with the program instructions,data files, data structures, and the like. The media may be continuouslystoring a program which may be executed with a computer, or temporarilystoring for execution or download. Also, the media may be variousrecording means or storing means in a form of single or a plurality ofhardware which are combined, but it is not limited to a media directlyaccessed to any computer system, and it may be distributed on network.Examples of the media include magnetic media such as hard disks, floppydisks, and magnetic tape; optical media such as CD ROM disks and DVD;magneto-optical media such as floptical disks; and hardware devices thatare specially configured to store and perform program instructions, suchas ROM (read-only memory), RAM (random access memory), flash memory, andthe like. Also, examples of other media include app store distributingapplications or recording media and storing media managed in sites,servers, and the like distributing other many software.

While certain example embodiments and implementations have beendescribed herein, other embodiments and modifications will be apparentfrom this description. Accordingly, the invention is not limited to suchembodiments, but rather to the broader scope of the presented claims andvarious obvious modifications and equivalent arrangements.

What is claimed is:
 1. A computer device, comprising at least one ormore memory devices comprising computer-readable instructions, and atleast one processor in communication with the one or more memorydevices, the at least one processor implemented to execute thecomputer-readable instructions to: manage interconnection relationbetween virtual space and physical space based on information of thevirtual space and information of the physical space, register a deviceexisting in the physical space as a digital object on the virtual space,provide a digital twin service based on a connection among theinformation of the virtual space, the information of the physical space,and the digital object on the virtual space, imitate an interactionbetween a user and the device in the physical space as an interactionbetween an agent corresponding to the user and the digital object in thevirtual space through the digital twin service, and determine a degreeof activation for the physical space based on the interaction betweenthe agent and the digital object in the virtual space; wherein the atleast one processor is further implemented to execute thecomputer-readable instructions to: register software in the digitalobject form of the virtual space by using a unique key value; and manageinformation of the physical space bound in the virtual space, and manageuser entry and visit records connected with the space.
 2. The computerdevice of claim 1, wherein the at least one processor is furtherimplemented to execute the computer-readable instructions to synchronizeuser information and activity information between the virtual space andthe physical space depending on whether synchronization of a user ispermitted or not.
 3. The computer device of claim 1, wherein the atleast one processor is further implemented to execute thecomputer-readable instructions to connect event information related tothe digital object of the virtual space as the information of thevirtual space.
 4. The computer device of claim 1, wherein the at leastone processor is further implemented to execute the computer-readableinstructions to connect information related to an agent of the virtualspace as the information of the virtual space.
 5. The computer device ofclaim 1, wherein the at least one processor is further implemented toexecute the computer-readable instructions to connect informationrelated to currency of the virtual space as the information of thevirtual space.
 6. The computer device of claim 1, wherein the at leastone processor is further implemented to execute the computer-readableinstructions to register a device recognizable in the virtual space byusing a code issued through device information or precertification. 7.The computer device of claim 1, wherein the at least one processor isfurther implemented to execute the computer-readable instructions toregister space recognizable in the virtual space by using map-based dataor space information.
 8. The computer device of claim 1, wherein the atleast one processor is further implemented to execute thecomputer-readable instructions to register a group consisting of aplurality of agents in the digital object form of the virtual space. 9.The computer device of claim 1, wherein the at least one processor isfurther implemented to execute the computer-readable instructions toregister a device in the form of precertification at the time of deviceproduction in case of the device providing a connection service with thevirtual space.
 10. The computer device of claim 1, wherein the at leastone processor is further implemented to execute the computer-readableinstructions to perform a status check of the device based on dataperiodically received from the device or information collected in thevirtual space.
 11. The computer device of claim 1, wherein the at leastone processor is further implemented to execute the computer-readableinstructions to provide a connection with a network service for traffictransmitted from the physical space.
 12. The computer device of claim 1,wherein the at least one processor, buffer or queue data by receivingthe data transmitted from the device or the space, and rout the data.13. The computer device of claim 1, wherein the at least one processoris further implemented to execute the computer-readable instructions tomanage local information and owner information related to the space asthe information of the physical space.
 14. The computer device of claim1, wherein the at least one processor is further implemented to executethe computer-readable instructions to issue and manage a key used for aconnection service between the virtual space and the physical space. 15.The computer device of claim 1, wherein the at least one processor isfurther implemented to execute the computer-readable instructions tomanage permission by each individual service subject participating inthe connection service between the virtual space and the physical space.